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Department  of  Defense  acquisition  poiicy  used  to  treat  the  modification  of  existing 
systems  to  meet  new  or  changing  requirements  as  an  aberration  rather  than  the 
norm.  Littie  attention  was  paid  to  management  of  iterative  processes  such  as 
prepianned  product  improvement,  spirai  deveiopment,  or  evoiutionary  acquisition. 
The  recent  shift  in  poiicy  toward  evoiutionary  acquisition,  as  the  preferred  approach 
to  meeting  operationai  requirements,  wiii  require  a  paradigm  shift  regarding 
management  of  defense  systems  acquisition  programs.  The  Advanced  Program 
Management  Course  offered  at  the  Defense  Systems  Management  Coilege  must 
be  modified  to  support  that  shift. 


The  environment  in  which  defense 
systems  acquisition  occurs  has  un¬ 
dergone  significant  change.  The  end 
of  the  Cold  War  has  led  to  extreme  reduc¬ 
tions  in  defense  spending  in  the  United 
States  and  around  the  world.  Downsizing 
of  military  forces  and  consolidation  of 
the  defense  industry  here  and  abroad 
have  significantly  altered  the  structure  of 
the  public  and  private  sectors.  The 
confluence  of  budgetary  reductions,  ab¬ 
sence  of  a  consistent,  long-term,  singu¬ 
lar  military  threat,  and  expansion  of  the 
civil  marketplace  as  a  driving  force  be¬ 
hind  technology  has  resulted  in  recent 
Department  of  Defense  (DoD)  adoption 


of  an  evolutionary  acquisition  process  as 
the  preferred  method  for  meeting  opera¬ 
tional  requirements  (Gansler,  1999).  De¬ 
spite  recognition  of  the  need  to  do  things 
differently,  some  things  remain  unchanged 
and,  to  a  large  extent,  unrecognized. 

Real-world  defense  systems  acquisition 
problems  are  largely  nondeterministic  in 
their  behavior}  Decisions  concerning  the 
acquisition  process  and  its  products  can 
and  do  result  in  unanticipated  outcomes. 
This  is  true  regarding  problematic  situa¬ 
tions  encountered  in  implementing  systems 
acquisition  policy  as  well  as  in  efforts  to 
match  defense  system  capabilities  with 
operational  and  support  requirements. 
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Defense  systems  cost  American  tax-pay¬ 
ers  billions  of  dollars.  These  programs  pe¬ 
riodically  undergo  intense  media  scrutiny 
and  political  debate.  The  resulting  impact 
on  individual  acquisition  programs  can  be 
chaotic.  Effective  management  of  defense 
system  acquisition  under  these  conditions 
clearly  requires  a  high  degree  of  technical, 
business,  and  political  acumen.  More  im¬ 
portant,  it  also  requires  an  understanding 
of  the  demands  that  such  complicating  fac¬ 
tors  place  on  managerial  activities.  The  re¬ 
search  reported  herein  was  aimed  at  inves¬ 
tigating  the  extent  to  which  individuals  un¬ 
dergoing  systems  acquisition  management 
training  at  the  Defense  Systems  Manage¬ 
ment  College  (DSMC)  are  prepared  to  meet 
those  demands.^ 

While  this  article  focuses  on  the  chal¬ 
lenge  facing  acquisition  managers  in  the 
DoD,  that  challenge  is  not  unique.  It  is 
reflective  of  the  impact  that  technology 
has  had  on  society  at  large.  In  modern 
times,  technology  has  been  the  catalyst 
for  unprecedented  speed  and  magnitude 
of  changes  that  quickly  outstrip  society’s 
ability  to  keep  pace.  Humanity’s  frequent 
inability  to  effectively  manage  large-scale 
systems  is  readily  apparent.^  The  combi¬ 
nation  of  technical,  organizational,  and 
personal  perspectives  when  faced  with  de¬ 
sign  and  management  of  such  systems  re¬ 
sults  in  solutions  marked  by: 

•  overconfidence  in  current  technical 
knowledge; 

•  failure  to  recognize  interactions  among 
system  components  that  have  been 
designed  relatively  independently;  and 

•  failure  to  anticipate  people  problems 
and  human  responses  in  crises.  (Mitroff 
&  Tinstone,  1993.) 


Research  Context 


Problematic  Situations: 

A  Maher  of  Scale 

We  are  not  concerned  here  with  reso¬ 
lution  of  problems  that  can  be  categorized 
as  routine — those  that  require  limited 
mental  processing  and  whose  outcome  is 
readily  observable.  Rather,  we  are  inter¬ 
ested  in  problems  characterized  by  effects 
that  are  distant  from  causes  in  time  as  well 
as  in  space — problems  with  few,  if  any, 
obvious  trigger  points  that  can  be  used  to 
produce  significant  and  lasting  change. 
Milan  Zeleny  (1977)  recognized  the  role 
of  scale  in  what  we  will  refer  to  herein  as 
“problematic  situations”  when  he  wrote 
that  human  systems  management  is  not 
interdisciplinary  or  multidisciplinary,  it 
does  not  attempt  to  unify  scientific  disci¬ 
plines,  it  transcends  them.  Such  is  our 
view  of  complexity  and  the  cognitive  chal¬ 
lenges  it  presents  in  all  forms  of  human 
endeavor.  Those  challenges  are  of  such 
scale  that  a  trans-disciplinary  paradigm  is 
required  for  effective  problem  resolution. 

Complexih:  A  Maher  of  Perspective 

As  yet,  there  is  no  agreed-upon  explicit 
definition  of  complexity,  although  there 
are  various  operational  descriptions 
(Cambel,  1993).  This  research  effort  pro¬ 
ceeded  from  the  presumption  that  com¬ 
plexity  in  the  defense  systems  acquisition 
process  is  a  matter  of  perspective;  it  re¬ 
sides  within  the  observer,  not  the  system 
under  investigation.  It  is  the  observer’s 
inability  to  grasp  the  interplay  of  multiple 
factors  and  events  that  lead  to  “complex” 
problems,  issues,  or  systems.  We  believe 
there  is  strong  support  in  the  literature  for 
such  a  position  and  the  need  to  make  a 
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clear  distinction  between  use  of  the  word 
“complex”  as  an  adjective  and  our  focus 
on  the  word  “complexity”  as  a  noun.  In 
our  view,  complexity  is  a  result  and  not  a 
cause  of  confusion  regarding  the  system, 
situation,  or  issue  under  consideration. 

The  Role  of  the  Observer 

According  to  Fischer  (1991),  the  first 
to  emphasize  the  peculiar  situation  of  the 
observer  was  R.  J.  Boscovich  in  “De 
Spatio  et  Tempore''  written  in  1758.  In 
Fischer’s  words:  “Boscovich,  a  forerun¬ 
ner  of  Einstein,  was  the  first  to  propose 
that  the  world  must  be  described  relative 
to  an  observer.  Moreover,  Boscovich 
claimed  that  the  observer  can  never  ob¬ 
serve  the  world  as  it  is — only  the  inter¬ 
face  (or  difference)  between  him  and  the 
world.”  This  notion  of  the  observer’s  role 
was  central  to  America’s  preeminent  nine¬ 
teenth  century  philosopher,  Charles  Sand¬ 
ers  Peirce,  who  wrote  about  the  triadic 
relationship  between  object  and  subject 
or  “interpretant”  through  sign  or  symbol 
(Paynter,  1968;  Floopes,  1991). 

The  basic  triadic  act  is  naming — creat¬ 
ing  a  symbolic  bridge  between  subject  and 
object.  It  is  the  interpretant,  the  observer, 
who  constructs  the  bridge.  Without  an 
observer,  there  is  no  observation.  It  fol¬ 
lows  that  if  the  observer  names  the  ob¬ 
ject,  complexity  is  in  the  observer’s  mind, 
not  in  the  object  under  investigation.  De¬ 
spite  these  early  insights  regarding  the 
nature  of  complexity,  the  philosophical 
bent  growing  out  of  the  machine  age  re¬ 
sulted  in  an  overshadowing  emphasis  on 
objectivity  and  the  deterministic,  mecha¬ 
nistic,  and  reductionist  perspective  of  late 
nineteenth  and  early  twentieth  century  sci¬ 
ence.  Mitroff  and  Tinstone  (1993)  main¬ 
tain  that  separation  of  subject  and  object 


is  a  paradigm  that  underlies  much  of  the 
approach  to  physical,  social,  and  manage¬ 
ment  science  education  even  today. 

The  ideas  of  Boscovich  and  Peirce  re¬ 
garding  the  role  of  the  observer  resurfaced 
with  the  emergence  of  systems  science 
following  World  War  II.  Several  propo¬ 
nents  of  systems  science  did  take  specific 
note  of  the  observer’s  role  in  characteriz¬ 
ing  the  nature  of  systems.  C.  West  Church¬ 
man  (1968)  wrote  that  it  is  a  silly  and 
empty  claim  that  an  observation  is  objec¬ 
tive  if  it  resides  in  the  brain  of  an  unbi¬ 
ased  observer.  W.  Ross  Ashby  (1956)  de¬ 
fined  a  system  as  any  set  of  variables  that 
the  observer  or 
experimenter 
selects  from 
those  available 
on  the  “real  ma¬ 
chine.”  Accord¬ 
ingly,  any  sys¬ 
tem  definition  is 
only  a  model  of  reality  constructed  sub¬ 
ject  to  the  observer’s  limitations  of  pur¬ 
pose  and  thought.  Charles  Francois  (1997) 
refers  to  Fleinz  von  Foerster  as  originator 
of  the  statement  that  objectivity  is  the  cog¬ 
nitive  version  of  the  physiological  blind 
spot.  Robert  Rosen  (1977)  specifically 
states  that  complexity  is  in  the  eye  of  the 
observer. 

The  Demahds  of  Complexity 

The  aim  of  classical  nineteenth  century 
science  was  to  discover  in  all  systems  some 
underlying  simple  level  of  operation  where 
deterministic  and  time -reversible  laws  of 
nature  applied.  In  the  classical  perspec¬ 
tive,  there  was  a  clear-cut  distinction  be¬ 
tween  what  was  considered  to  be  simple 
and  what  had  to  be  considered  as  com¬ 
plex.  The  concept  of  complexity  within 


"The  basic  triadic 
act  is  naming — 
creating  a  symbolic 
bridge  between 
subiect  and  ebiect." 
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systems  thinking  has  evolved  considerably 
since  that  time  (Camhel  1993,  De  Greene 
1993,  Klir  1991.)  Several  schools  of 
thought  have  arisen  during  the  latter  half 
of  the  twentieth  century  to  address  the  man¬ 
agement  of  complexity  (Warfield,  1996a.) 

We  found  the  following  words  of 
Warfield  (1995a)  to  he  the  most  powerful 
reason  for  the  thrust  of  our  research  ef¬ 
fort.  They  clearly  identify  where  complex¬ 
ity  resides  and  underscore  the  need  for  a 
paradigm  shift  in  the  managerial  approach 
to  prohlematic  situations.  “To  misplace  the 
origin  of  complexity  in  the  object  of  in¬ 
quiry,  instead  of  in  the  mind  of  the  ob¬ 
server,  is  to  commit  an  error  that  is  un¬ 
likely  to  be  un- 
daie. . ..  If,  how¬ 
ever,  it  is  cor¬ 
rectly  realized 
that  complexity 
is  in  the  mind 
of  the  beholder, 
the  possibility 
of  reducing 
complexity 
through  learning  processes  comes  to  the 
fore.”  (Emphasis  added.)  Thus,  we  em¬ 
barked  upon  a  line  of  research  aimed  at 
identifying  cognitive  barriers  to  be  over¬ 
come  if  we  are  to  be  successful  in  un¬ 
derstanding  the  nature  of  complexity.  And, 
we  chose  to  pursue  that  research  in  an  edu¬ 
cational  institution  dedicated  to  improv¬ 
ing  the  systems  acquisition  management 
process. 

The  Research  Questioh 
AHD  Methodology 


We  sought  to  find  an  answer  to  the  fol¬ 
lowing  question:  Are  members  of  the 


defense  acquisition  workforce  prepared  to 
meet  the  demands  of  complexity? 

Input  for  analysis  was  gathered  through 
questionnaires  administered  to  highly 
schooled  engineering-  and  management- 
oriented  government  employees  respon¬ 
sible  for  the  acquisition  and  life-cycle  sup¬ 
port  of  defense  systems.  Virtually  all  par¬ 
ticipants  were  college  graduates  with  10 
or  more  years  of  on-the-job  experience. 
Most  held  bachelor’s  degrees  in  an  engi¬ 
neering  or  business  discipline  and  many 
held  master’s  degrees  as  well. 

Study  participants  included  acquisition 
professionals  attending  the  14-week 
Advanced  Program  Managers  Course 
(APMC)  and  members  of  the  faculty  at 
the  DSMC  located  at  Fort  Belvoir,  VA. 
The  College  is  considered  to  be  a  premier 
center  for  learning  about  the  DoD  systems 
acquisition  process.  Successful  comple¬ 
tion  of  APMC  is  considered  essential  for 
selection  as  a  program  manager  of  a  major 
defense  systems  acquisition  program. 

The  principal  research  effort  comprised 
three  separate  studies  conducted  between 
January  1996  and  February  1999.  More 
than  875  individuals  from  seven  acquisi¬ 
tion  management  courses  participated  in 
the  research  project.  A  combination  of 
content  analysis  as  described  by  Weber 
(1990)  and  nonparametric  statistical  anal¬ 
ysis  as  described  by  Siegel  and  Castellan 
(1988)  was  selected  as  an  appropriate  set 
of  procedures  for  analyzing  most  partici¬ 
pant  responses  to  self-administered  survey 
instruments.  Random  sampling  and  infer¬ 
ential  statistical  analytical  techniques  were 
applied  to  the  extent  practical.  Significant 
reliance  on  nonrandom  purposive  sam¬ 
pling  permit  us  to  describe  what  was  dis¬ 
covered,  but  not  to  state  generalizable  con¬ 
clusions  concerning  the  associations  or 


"Mere  than  875 
individuals  from 
seven  acquisition 
management 
courses  participated 
in  the  research 
prelect." 
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patterns  uncovered.  This  restriction  was 
deemed  acceptable  since  participant 
demographics  generally  reflect  the  com¬ 
position  of  the  DoD  acquisition  workforce. 

Results 


Barriers  to  the  Interpretation 
OF  Structural  Graphics 

The  first  study  focused  on  interpreta¬ 
tion  of  graphical  displays  designed  to  aid 
in  the  management  of  problematic  situa¬ 
tions.  Such  situations  are  comprised  of 
multiple  interrelationships  difficult  to  con¬ 
vey  through  text  alone.  To  enhance  the 
practical  benefits  of  this  research  effort, 
we  chose  to  use  graphical  displays  noted 
for  their  track  record  as  viable  manage¬ 
ment  tools.  A  set  of  graphical  displays 
known  as  interpretive  structural  models, 
met  this  requirement.  They  are  the  prod¬ 
ucts  of  a  process  called  interactive  man¬ 
agement  (IM)  developed  by  John  N. 
Warfield,  a  pioneer  in  the  management  of 
complexity  through  systems  design.  The 
IM  process,  products,  and  scientific  foun¬ 
dations  are  described  in  the  many  publi¬ 
cations  of  Warfield  and  his  colleagues 
(Warfield,  1990,  1996b). 

Graphical  displays  can  be  an  extremely 
efficient  means  of  communication  if  the 
viewer  understands  the  rules  of  construc¬ 
tion.  However,  rules  for  proper  construc¬ 
tion  of  interpretive  structural  graphics  are 
not  easy  to  articulate.  Furthermore,  visual 
skills,  unlike  talking,  reading,  and  writ¬ 
ing  skills,  have  been  left  dangling  by  our 
Western  educational  system  (Eisner, 
1993).  To  presume  intuitive  understand¬ 
ing  of  graphical  displays  is  erroneous. 
Research  in  the  field  of  visual  literacy 
points  out  that  while  looking  may  be  a 


given,  seeing  and  understanding  is  an 
achievement  (Feinstein  &  Hagerty,  1994). 

We  limited  our  investigation  to  an  in¬ 
terpretive  structural  model  designed  to 
facilitate  problem  definition  and  resolu¬ 
tion.  It  is  the  model  most  often  developed 
first  in  the  IM  process  and  the  one  most 
often  subject  to  misinterpretation  by  first¬ 
time  viewers.  The  graphical  display  of  this 
model  is  called  the  “problematique” 
(Warfield  &  Perino,  1999).  The  purpose 
of  this  study  was  twofold:  first,  to  iden¬ 
tify  common  misperceptions  of  the 
problematique  among  first- time  viewers; 
and  second,  to 
identify  likely 
causes  for  their 
misinterpreta¬ 
tion  of  graphical 
syntax.  It  was 
anticipated  that 
such  informa¬ 
tion  would  fa¬ 
cilitate  develop¬ 
ment  of  educational  material  aimed  at 
increasing  viewer  comprehension.  More 
than  475  acquisition  professionals  partici¬ 
pated  in  this  research. 

Results  showed  significant  misinter¬ 
pretation  of  the  problematique.  Partici¬ 
pants  had  little  or  no  prior  training  in 
the  use  of  this  display.  But  their  inability 
to  correctly  interpret  the  display’s  format 
and  underlying  logic,  even  when  written 
instructional  material  was  provided,  was 
surprising.  The  average  score  among  the 
170  respondents  asked  to  interpret  a 
problematique  without  benefit  of  instruc¬ 
tion  was  22  percent.  The  average  score 
among  314  respondents  with  access  to 
written  instruction  was  only  45  percent. 
Analysis  of  narrative  responses  to  ques¬ 
tions  about  the  meaning  of  the  display  led 


"Graphical  displays 
can  be  an  extremely 
efficient  means  of 
communication  if 
the  viewer  under¬ 
stands  the  rules 
of  construction." 
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us  to  conclude  that  participants  were  pre¬ 
disposed  to  reductive  reasoning  and  em¬ 
phasis  on  cause  and  effect  as  a  principal 
mode  of  thought.  To  the  extent  that  this 
conclusion  is  valid,  it  provides  cause  for 

concern  regard¬ 
ing  effective 
management  of 
the  DoD  sys¬ 
tems  acquisi¬ 
tion  process. 
That  process  is 
lengthy  and 
complicated.  It 
is  subject  to 
technical  as 
well  as  politi¬ 
cal  perturbations.  Both  the  process  and 
its  products  are  socio-technical  in  nature. 
As  such,  they  are  emergent,  not  mecha¬ 
nistic  in  behavior.  Taking  management 
action  based  on  a  paradigm  of  determin¬ 
ism  invites  repeated  failures  in  program 
execution  and  a  terrible  waste  of  national 
resources. 

Managerial  Assumptions  about 
THE  Nature  of  Complexity 

The  second  study  focused  on  acquisi¬ 
tion  workforce  participant  opinions  regard¬ 
ing  the  nature  of  complexity.  Warfield 
(1998)  identified  a  series  of  assumptions 
he  believes  people  make  about  the  nature 
of  complexity.  He  believes  these 
assumptions  interfere  with  the  effective 
management  of  large-scale  problematic 
situations  to  such  a  degree  that  he  has  la¬ 
beled  them  as  “killer  assumptions.” 
Warfield  also  identified  a  series  of  demands 
that  complexity  places  on  management. 
The  demands  of  complexity  are  the  antith¬ 
esis  of  the  killer  assumptions.  The  purpose 
of  this  second  study  effort  was  to  assess 


how  widely  each,  if  any,  of  the  killer  as¬ 
sumptions  might  be  held  among  individu¬ 
als  responsible  for  managing  the  acquisi¬ 
tion  and  life-cycle  support  of  national  de¬ 
fense  systems.  This  study  included  85 
APMC  students  and  28  faculty  at  DSMC 
and  was  completed  in  December  1998. 
Killer  assumptions  and  their  antithetical 
demands  of  complexity  are  listed  side  by 
side  in  Table  1 .  The  percentage  of  respon¬ 
dents  selecting  each  alternative  is  shown 
in  parentheses. 

The  total  number  of  killer  assumption 
statements  chosen  by  respondents  is  dis¬ 
played  in  Figure  1 .  Totals  ranged  from  a 
low  of  0  to  a  high  of  14.  The  most  fre¬ 
quent  number  of  killer  assumption  state¬ 
ments  chosen  by  any  one  individual  was 
three.  The  average  number  was  four. 

The  four  most  frequently  chosen  killer 
assumption  statements  deal  with  confidence 
in  human  learning  powers,  the  location  of 
complexity,  belief  in  the  intellectual  capac¬ 
ity  of  executives,  and  the  assumption  that 
natural  language  is  adequate  to  represent 
complexity  (Table  1).  In  combination, 
these  four  suggest  that  resolution  of  large- 
scale  problems  presents  no  unique  chal¬ 
lenge.  The  two  most  frequently  combined 
killer  assumptions  were  that  human  learn¬ 
ing  powers  are  independent  of  what  is  to 
be  learned  and  that  complexity  is  in  the 
system  being  ob-served.  This  is  worri¬ 
some,  as  it  indicates  that  overcoming  cog¬ 
nitive  barriers  to  the  management  of  prob¬ 
lematic  situations  will  be  a  daunting  task. 
It  was  encouraging  to  find  that  faculty  were 
not  as  likely  to  choose  killer  assumption 
statements  as  were  the  course  attendees. 
As  Figure  2  shows,  the  maximum  num¬ 
ber  of  killer  assumptions  chosen  by  any 
one  faculty  member  was  six  and  the  most 
frequent  number  chosen  was  three. 


"Taking  manage¬ 
ment  action  based 
on  a  paradigm  of 
determinism  invites 
repeated  failures  in 
program  execution 
and  a  terrible 
waste  of  national 
resources." 
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Table  1 .  Percentage  of  Respondents  Cheesing  Killer  Assumptions 
versus  Antithetical  Demands  of  Complexity 


Killer  Assumptions 

Demands  of  Complexity 

Human  learning  powers  are 
independent  of  the  scale  of  what  is 
to  be  learned. 

(62  percent) 

Individuals  cannot  resolve  complexity 
simply  by  thinking  about  it  or  discussing 
it  in  unorganized  group  conversation. 

(38  percent) 

The  site  of  complexity  is  in  the  system 
being  observed. 

(46  percent) 

The  complexity  of  a  situation  is  distrib¬ 
uted  among  many  minds. 

(54  percent) 

The  executive  has  the  intellectual 

capacity  to  comprehend: 

•  all  of  the  factors  that  are  relevant  to 
an  executive  decision; 

•  how  the  various  factors  are  inter¬ 
related  in  a  problematic  situation; 

•  what  alternatives  are  relevant  when 
it  is  time  to  make  a  choice; 

•  how  to  prioritize  the  alternatives; 
and 

•  at  what  time  action  should  be 
initiated. 

(44  percent) 

Complexity  demands  that  organizations 
accept  the  inevitability  of  executive 
inadequacy  to  resolve  complexity,  as  an 
inherent  property  of  every  human  being. 

(56  percent) 

Natural  language  is  adequate  to 
represent  complexity. 

(44  percent) 

The  inadequacy  of  natural  language 
(e.g.,  linearity)  must  be  recognized; 
graphical  nonlinear  logic  must  be  widely 
adopted  in  the  domains  of  complexity  to 
help  overcome  that  inadequacy. 

(56  percent) 

Representation  of  complexity  through 
metaphors  related  to  common 
quantitative  formalisms  from  physical 
sciences  is  strongly  contributory  to 
the  resolution  of  complexity. 

(36  percent) 

Complexity  demands  portrayal  of  the 
logic  underlying  the  problematic 
situation. 

(64  percent) 

There  is  no  reason  to  provide  any 
special  infrastructure  at  work  to  deal 
with  complexity. 

(30  percent) 

A  workplace  infrastructure  dedicated  to 
resolving  complexity  would  satisfy  a 
major  demand  of  complexity. 

(70  percent) 
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Table  1 .  Percentage  of  Respondents  Cheesing  Killer  Assumptions 
versus  Antithetical  Demands  of  Complexity  (continued) 


Killer  Assumptions 

Demands  of  Complexity 

Academics  should  be  free  to  call  any 
subject  that  they  choose  a  “science” 
with  no  institutionally  established 
requirements  and  standards  for 
linguistic  quality  control. 

(23  percent) 

The  word  “science”  must  be  restricted  to 
those  fields  in  which  archival  history, 
established  laws,  adequate  empirical 
evidence,  and  adequate  metrics  have 
been  established  to  form  a  science. 

(77  percent) 

The  extent  of  valid  application  of 
common  quantitative  formalisms  from 
physical  sciences  into  socio-technical 
arenas  is  very  large,  and  can  be 
organized  so  that  it  is  almost 
automatic. 

(23  percent) 

Advocacy  of  unvalidated  metaphors  of 
formalisms  from  physical  science  must 
be  tempered;  justification  and  empirical 
evidence  must  be  provided  to  support 
such  advocacy. 

(77  percent) 

There  is  no  need  to  allocate  space 
specifically  for  the  purpose  of 
portraying  complexity. 

(23  percent) 

Complexity  demands  that  workspace 
allocation  be  designed  especially  to 
facilitate  human  learning. 

(77  percent) 

Normal  processes  are  sufficient  to 
enable  description  and  diagnosis  of 
problematic  situations  involving  high 
complexity. 

(22  percent) 

The  design  of  group  processes  must 
suit  the  character  of  complexity,  rather 
than  simply  using  conventional 
processes  or  allowing  NO  process 
design. 

(78  percent) 

It  is  appropriate  to  discuss  science 
and  technology  as  though  there  are 
no  essential  distinctions  between 
them. 

(21  percent) 

Complexity  demands  that  technology 
used  to  help  resolve  problematic 
situations  shall  have  been  founded  in 
science,  and  not  just  imposed  by  highly 
vocal  advocates. 

(79  percent) 

There  is  no  need  for  empirical 
evidence  to  justify  assumptions  of 
relevance  when  designing  processes 
to  support  resolution  of  complexity. 

(17  percent) 

Scientifically  respectable  evidence  must 
be  applied  in  designing  processes  to 
support  resolution  of  complexity. 

(83  percent) 
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Table  1 .  Percentage  of  Respondents  Cheesing  Killer  Assumptions 
versus  Antithetical  Demands  of  Complexity  (continued) 


Killer  Assumptions 

Demands  of  Complexity 

Simple  amalgamation  of  disciplines 
will  relieve  disciplinary  shortcomings 
mands 

in  considering  comprehensive 
domains. 

(15  percent) 

Interdisciplinary  programs  must  be 
designed  to  meet  complexity’s  de¬ 
fer  learning  efficacy. 

(85  percent) 

There  is  seldom  any  reason  to  give 
the  choice  of  types  of  relationships 
that  are  to  be  used  in  studies  the 
same  level  of  effort  and  depth  of 
selectivity  that  are  given  to  the 
elements  that  will  be  related 
(e.g.,  in  model  development). 

(14  percent) 

In  problematic  situations,  the  choice  of 
relationships  to  be  applied  shall  have  as 
much  prominence  in  the  thinking  of 
practitioners  as  does  the  choice  of 
elements  that  are  to  be  related. 

(86  percent) 

The  findings  from  behavioral  science 
about  individuals,  groups,  and 
organizations  are  too  “soft”  to  have 
a  major  role  in  the  management  of 
organizations. 

(11  percent) 

Linkages  between  thought  leaders  from 
the  past  and  practices  invoked  in 
organizations  must  be  widely  under¬ 
stood,  and  taken  into  account  in  self¬ 
regulation  of  human  behavior. 

(89  percent) 

If  information  comes  from  a 
“prestigious”  source,  it  need  not  be 
questioned. 

(5  percent) 

The  authority  of  “prestigious  institutions” 
must  be  tested  against  the  scientific 
base  that  ought  to  be  provided  to 
support  that  authority. 

(95  percent) 

There  is  no  need  for  empirical 
evidence  to  justify  assumptions  of 
relevance  when  designing  processes 
to  support  resolution  of  complexity. 

( 3  percent) 

Scientifically  respectable  evidence  must 
be  applied  in  designing  processes  to 
support  resolution  of  complexity. 

(97  percent) 
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It’s  important  not  to  discount  Warfield’s 
hypothesis  that  educational  institutions  fail 
to  prepare  students  to  deal  adequate-ly  with 
the  demands  of  complexity  (Warfield, 
1995h,  1996c).  This  is  particularly  so  given 
the  apparent  level  of  faculty  confidence 
in  human  cognitive  abilities.  More  than 
60  percent  of  the  faculty  participants  in 
this  study  agreed  with  the  statement  that 
human  learning  powers  are  independent 
of  the  scale  of  what  is  to  he  learned.  Yet, 
the  inability  of  the  human  mind  to  pro¬ 
cess  more  than  a  few  bits  of  information 
simultaneously  is  well  known  (Waller, 
1982;  Miller,  1956;  Simon,  1974; 
Warfield,  1988).  The  tendency  is  to 
underconceptualize  interrelationships, 
thereby  avoiding  cognitive  overload. 
Underconceptualization  leads  to  insuffi¬ 
cient  understanding  of  problematic  issues 
by  any  single  individual  or  group  of  indi¬ 
viduals  with  all  the  unfortunate  outcomes 
that  result  (Warfield,  1991). 

The  Nature  of  Systems 
AND  Problem  Solving 

The  third  study  involved  more  than  300 
acquisition  professionals  and  focused  on 
obtaining  their  opinions  regarding  the 
nature  of  systems  and  problem  solving. 
Results  of  the  first  two  studies  had  led  us 
to  wonder  about  participants’  perspective 
regarding  systems  theory.  As  previously 
stated,  most  survey  participants  held  un¬ 
dergraduate  and  graduate  degrees  in  en¬ 
gineering  or  business  management  sub¬ 
jects.  The  curriculum  of  the  systems  ac¬ 
quisition  management  course  they  were 
attending  addressed  both  theory  and  prac¬ 
tice  in  systems  management  tools  and 
techniques,  yet  survey  responses  had  of¬ 
ten  reflected  a  simplistic  approach  to  prob¬ 
lem  solving.  We  therefore  determined  that 


important  insights  about  this  phenomenon 
could  be  gained  by  obtaining  APMC  at¬ 
tendee  responses  to  the  following  three 
open-ended  questions: 

•  What  definition  of  “system”  do  you 
think  is  most  useful? 

•  What  does  “problem  solving”  involve? 

•  How  might  “system  behavior”  be  best 
understood? 

Input  was  obtained  by  administering  a 
one -page  questionnaire  and  applying  con¬ 
tent  analysis  procedures  to  the  responses. 
Analysis  disclosed  a  predominantly 
Newtonian  perspective  among  the  partici¬ 
pants. Well  over  half  the  respondents  felt 
that  system  behavior  could  be  best  under¬ 
stood  through  observation  and  analysis. 
Almost  the  same  proportion  described  a 
problem-solving  process  that  did  not  in¬ 
clude  getting  feedback  to  determine  if  the 
chosen  solution  was  working.  These  re¬ 
sults  gave  weight  to  concerns  raised  dur¬ 
ing  our  earlier  studies  that  acquisition  pro¬ 
fessionals  attending  APMC  were  overly 
focused  on  near-term  observable  out¬ 
comes  and  decision  making,  and  less  so 
on  actual  problem  solving. 

Aggregate  Findings 


This  research  effort  sought  to  answer 
this  question:  Are  members  of  the  defense 
acquisition  workforce  prepared  to  meet  the 
demands  of  complexity?  Three  studies  were 
conducted.  The  purpose  of  the  first  was  to 
determine  if  first-time  viewer  comprehen¬ 
sion  of  a  problematique  can  be  improved 
by  providing  written  instruction.  The  pur¬ 
pose  of  the  second  was  to  determine  which 
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if  any  of  Warfield’s  killer  assumptions  are 
widely  held  among  defense  systems  ac¬ 
quisition  professionals.  The  purpose  of  the 
final  study  was  to  gain  insight  to  acquisi¬ 
tion  professionals’  view  of  systems  man¬ 
agement.  Combining  the  results  of  those 
three  studies  led  us  with  the  following 
aggregate  findings: 

•  Acquisition  professionals  share  a  pre¬ 
disposition  for  reductive  reasoning  and 
a  reliance  on  a  simplistic  linear  approach 
as  a  principal  mode  for  managerial 
actions  (studies  1  and  3). 

•  Acquisition  professionals  are  very  con¬ 
fident  of  the  capacity  for  human  learn¬ 
ing  regardless  of  the  scale  of  what  is 
to  he  learned  (study  2). 

•  There  is  considerable  difference  of 
opinion  among  acquisition  profession¬ 
als  concerning  whether  the  site  of  com¬ 
plexity  is  intrinsic  to  a  system  under 
observation  or  resident  in  the  mind  of 
the  observer  (study  2). 

Conclusions  and  Recommendations 


Results  of  the  research  support  a  con¬ 
tention  that  defense  systems  acquisition 
professionals  represented  by  the  research 
sample  are  not  adequately  prepared  to  deal 
with  the  demands  of  complexity  when  at¬ 
tempting  to  manage  the  nondeterministic 
aspects  of  large-scale  systems  acquisition 
programs.  The  prevailing  strategy  for  sys¬ 
tems  acquisition  is  Newtonian  in  its  ori¬ 
gin  and  linear  in  its  essential  character¬ 
istics.  It  embodies  analysis  and  control 
of  observable  outcomes  and  drives  mana¬ 
gerial  attention  toward  near-term  time 
horizons.  Such  a  strategy  may  be 


appropriate  for  well-defined  mechanistic 
systems,  but  is  inappropriate  when  attempt¬ 
ing  to  manage  problematic  situations  en¬ 
countered  during  the  defense  systems  ac¬ 
quisition  process. 

The  curriculum  of  the  APMC  at  DSMC 
follows  DoD  policy  that,  until  recently, 
emphasized  a  linear  flow  of  activities  start¬ 
ing  from  the  establishment  of  war  fighter 
requirements,  through  systems  develop¬ 
ment,  production,  life  cycle  support,  and 
ending  with  disposal.  Relatively  little  at¬ 
tention  has  been  paid  to  challenges  faced 
when  existing  socio-technical  systems 
must  be  modified  to  meet  new  or  chang¬ 
ing  requirements.  Iterative  processes  such 
as  preplanned  product  improvement  (P^I), 
spiral  development,  or  evolutionary  ac¬ 
quisition  have  been  treated  as  aberrations 
rather  than  as  the  norm.  Such  an  approach 
flies  in  the  face  of  reality  given  current 
emphasis  on  extending  the  life  of  existing 
systems  and  the  evolutionary  development 
of  new  systems.  There  is  a  pressing  need 
for  a  paradigm  shift  regarding  manage¬ 
ment  of  defense  systems  acquisition  pro¬ 
grams.  Curriculum  that  places  undue  em¬ 
phasis  on  objectivity  and  the  determinis¬ 
tic,  mechanistic,  and  reductionist  perspec¬ 
tive  of  late  nineteenth  and  early  twentieth 
century  science  must  be  replaced  with  one 
that  prepares  graduates  to  operate  within 
the  nonlinear,  nondeterministic  reality  of 
large-scale  socio-technical  systems.  The 
following  actions  are  recommended  for 
DSMC  management  to  help  bring  about 
the  transformation: 

•  Make  provisions  for  educating  faculty, 
staff,  and  students  regarding  the 
demands  of  complexity  to  include 
learning  theory  and  human  learning 
capacity  under  varying  conditions. 
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•  Increase  emphasis  within  the  curric¬ 
ulum  on  the  use  of  science-hased  and 
empirically  tested  methods  for 
resolving  prohlematic  situations. 

•  Augment  the  functional  faculty  orga¬ 
nization  to  facilitate  a  transdisciplin- 
ary  approach  to  the  application  of 
management  and  leadership  principles. 

These  actions  will  help  bring  about  the 
educational  transformation.  But  a  total 
transformation  will  be  difficult  as  long  as 
acquisition  professionals  continue  to  over¬ 
estimate  human  cognitive  ability  to  con¬ 
tend  with  large-scale  problematic  situa¬ 
tions  and  fail  to  recognize  that  complex¬ 
ity  is  endemic  to  the  observer  rather  than 
an  intrinsic  system  characteristic. 

The  findings,  conclusions,  and  recom¬ 
mendations  presented  herein  pertain  spe¬ 
cifically  to  members  of  the  defense  sys¬ 
tems  acquisition  workforce  attending  the 
14-week  APMC  at  DSMC  but  could  pos¬ 
sibly  be  generalized  across  the  entire 
acquisition  workforce.  APMC  students 
may  be  unique  in  the  level  of  education 
and  experience  they  bring  to  the  academic 
environment,  but  they  are  the  products  of 
America’s  educational  institutions.  There 
is  abundant  evidence  from  this  study  to 
suggest  the  need  for  research  regarding 
the  educational  paradigm  underlying 
engineering  and  business  management 
education  in  the  United  States.  Results  of 
such  research  may  identify  a  need  for  or¬ 
ganizational  change  and  curriculum  reform 
within  our  colleges  and  universities  to  pro¬ 
duce  graduates  able  to  meet  the  demands 
of  complexity  as  they  attempt  to  resolve 
the  large-scale  socio-technical  problems 
facing  our  nation. 


Postscript 


A  survey  of  acquisition  professionals 
attending  the  APMC  and  graduates  of  the 
Executive  Program  Managers  Course 
(EPMC)  was  conducted  during  March  and 
April  of  2000.  This  survey  focused  on  the 
four  “killer  assumption”  statements  most 
frequently  chosen  by  prior  respondents  and 
the  three  open-ended  questions  about  the 
nature  of  systems  and  problem  solving. 
Those  tend  to  reinforce  the  recommenda¬ 
tions  being  made  herein. 

Eor  example,  the  APMC  survey  par¬ 
ticipants  were  attending  the  latter  stages 
of  a  revised  14-week  course  designed  to 
increase  emphasis  on  “real  world”  acqui¬ 
sition  problems  with  a  problem-solving 
model  that  included  a  feedback  loop.  Yet 
only  39  percent  of  these  survey  respon¬ 
dents  mentioned  getting  feedback  when 
they  responded  to  the  question:  What  does 
“problem  solving”  involve?  In  response 
to  the  same  question,  only  24  percent  of 
respondents  among  EPMC  graduates  in¬ 
cluded  obtaining  feedback  on  imple¬ 
mented  solution  in  their  answer.  Although 
the  EPMC  graduates  did  not  have  the  ben¬ 
efit  of  attending  the  revised  APMC,  such 
a  potentially  low  attention  to  follow-up 
among  major  DoD  systems  acquisition 
program  managers  and  deputy  program 
managers  is  quite  worrisome.  These  re¬ 
sults  suggest  a  need  for  increased  empha¬ 
sis  on  post-decision  follow-up  as  part  of 
the  problem-solving  process  taught  at 
DSMC. 

In  addition,  belief  in  human  learning 
powers  was  very  strong  in  both  groups. 
Seventy-two  percent  of  APMC  attendees 
and  67  percent  of  EPMC  graduates  fa¬ 
vored  the  “killer  assumption”  that  human 
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learning  powers  are  independent  of  the 
scale  of  what  is  to  he  learned.  While  this 
response  may  he  biased  due  to  associa¬ 
tion  with  the  DSMC  educational  environ¬ 
ment,  it  does  suggest  the  possibility  of 
over-confidence  regarding  the  capacity  for 
dealing  with  large-scale  systems.  In  this 

regard,  it  is  in¬ 
teresting  to 
note  that  the 
APMC  respon¬ 
dents  had  a 
higher  regard 
for  executive 
capabilities 
than  EPMC 
graduates  did. 
Forty-nine  per¬ 
cent  of  the 
APMC  respon¬ 
dents,  as  opposed  to  33  percent  of  the 
EPMC  respondents,  favored  the  follow¬ 
ing  “killer  assumption”  statement:  The 
executive  has  the  intellectual  capacity  to 
comprehend: 


•  all  of  the  factors  that  are  relevant  to  an 
executive  decision; 

•  how  the  various  factors  are  interrelated 
in  a  problematic  situation; 

•  what  alternatives  are  relevant  when  it 
is  time  to  make  a  choice; 

•  how  to  assign  priority  to  the  alternatives; 
and 

•  at  what  time  action  should  be  initiated. 

Since  most  EPMC  graduates  have  also 
attended  APMC,  one  could  hypothesize 
that  increased  “real  world”  experience  may 
tend  to  reduce  an  individual’s  faith  in  their 
own  ability  to  manage  problematic  situa¬ 
tions.  These  results  suggest  that  the  cur¬ 
rent  effort  to  include  more  “real  world” 
learning  experiences  in  the  APMC  cur¬ 
riculum  is  a  step  in  the  right  direction. 


"...one  could 
hypothesize  that 
increased  "real 
world"  experience 
may  lend  to  reduce 
an  individual's 
faith  in  their  own 
ability  to  manage 
prehlemalic 
situations." 
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Endnotes 


1 .  A  growing  body  of  literature  provides 
ample  support  for  such  a  premise. 
See,  for  example,  Cambel  (1993),  De 
Greene  (1993),  Kiel  (1994),  and 
Waldrop  (1992)  for  contemporary 
thoughts  by  systems  thinkers.  See  also 
Fenster  (1999)  for  a  telling  example 
of  an  acquisition  management  disas¬ 
ter  within  the  Department  of  Defense. 

2.  The  Defense  Acquisition  Workforce 
Improvement  Act  (DAWIA),  Public 
Law  101-510,  Title  10  U.S.C.,  was 
enacted  to  improve  the  effectiveness 
of  the  personnel  who  manage  and 
implement  defense  acquisition  pro¬ 
grams.  As  part  of  the  fiscal  year  1991 
Defense  Authorization  Act,  it  called 
for  establishing  an  Acquisition  Corps 
and  professionalizing  the  acquisi¬ 
tion  workforce  through  education, 
training,  and  work  experience. 


3.  Socio-technical  system  failures  such 
as  Chernobyl,  Challenger,  and  fratri¬ 
cide  in  the  Gulf  War  provide  tragic 
reminders  of  large-scale  system  mis¬ 
management.  Similar  failures,  with 
arguably  less  tragic  consequences,  are 
reported  almost  daily  by  the  world 
news  media. 

4.  By  Newtonian,  we  mean  an  investi¬ 
gative  approach,  born  in  the  17*  cen¬ 
tury,  that  proved  successful  working 
with  systems  characterized  by  a  very 
small  number  of  variables,  a  high  de¬ 
gree  of  determinism,  and  suitable  for 
analytical  treatment.  Problems  with 
such  characteristics  have  been  referred 
to  as  problems  of  organized  simplic¬ 
ity  (Klir,  1985). 
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